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Today’s Presentation 


INCOSE 


• Illustrates some key strategic aspects for conducting effective 
concept design & design-to-cost trade studies 

> What concept design is & why it’s important 

> Fidelity needed in concept design solution 

> Techniques in designing mission level trade space 

> Challenges in determining credible design convergence 

> Recommended practices 
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Important Note 


INCOSE 


• Concept design may be conducted using a variety of methods 

• This presentation describes seiected aspects of one method for 
conducting a concept design study 

> Uses space observatory example 

> Best suited to immature mission concepts that advance state of the art 
and that have high design uncertainty 
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What Concept Design is 

& 


Why it’s Important 
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Concept Design is Expioratory Process to \ ncOSE 
Determine System Levei Design Baseiine 

• Conducted in pre-Phase A & Phase A of Project Life Cycie to 
provide ‘"feasibie” system ievei design baseiine for new concept 

• As much an investigation of requirements as of design 


> Concurrent investigation of: 

□ Concept of operations 

□ Requirements 

□ Design 

□ Performance 

□ Technology development 

□ Verification approach 

□ Flight dynamics 

□ Ground segment (ground stations, mission & science ops centers) 

□ Launch interface 

□ Cost 


□ Schedule 

□ Risks, etc. 
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Concept Design Performed in Pre-Phase A | nCOSE 
& Phase A of NASA Project Life Cycie . ! ’ll- 

Figure 1 
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Concept Design Plays Central Role in 
Project Success 


INCOSE 


• Earliest life cycle phases have most leverage over life cycle cost 


> Concept design product effectively locks (or renders unchangeable) the 
majority of system LCC 

• Such extraordinary leverage presents business case for 
conducting concept design in pragmatic & rigorous fashion 

> Particularly important for immature mission concepts that advance state 
of the art and that have high design uncertainty 


(LCC) 
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Figure 2 



Conceptual illustration from ref. (c), adapted for presentation 
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Concept Design Plays Central Role in \ ncOSE 
Project Success (Cont’d) -3 ’ 


• Done well, provides executable system level design baseline for 
project teams in Phase B & later phases 

• Not done well, can subject project teams in Phase B & later 
phases to system level redesign - in some cases, to multiple 
system level redesigns accompanied by: 

> Fluid technical baselines with ever-decreasing capabilities 

> Cost overruns & recurring schedule delays 

> Contract disputes & cancellations 

> Challenges in retaining trained personnel 
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Pre-Phase A / Phase A Offer Unique Venue | n COS E 
for System Level Trades " ’ 


• Teams small, agile, closely coordinated 

> Typically operate absent many formalities of later project phases 

□ e.g., typically no prime contracts, system level requirements not under 
configuration control until late in phase A 

> Can accommodate high rate of change in system level “requirements’’ & 
design characteristics (R&DC) 

□ Enables broad investigation of trade space in relatively short time 

• Note: 

> “requirements” in quotes denotes interim reference capabilities used to 
guide evaluation of point designs in trade space 

> System level requirements aren’t baselined until SRR for a final concept 

design that meets technical & programmatic (including cost & schedule) 
constraints anniversary 
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Phase B & Later Development Phases Not \ ncOSE 
Well Suited for System Level Trades I ’ 


• In Phase B, system level design is more difficult & expensive to 
change, e.g., 

> Teams typically larger & more distributed 

> Prime contracts typically in place 

> System level requirements typically under configuration control 

> Preliminary design work assumes system level design complete 

• In Phases C & D, system level changes even more difficult & 
expensive to change 

> Teams typically even larger than in Phase B 

> System & subsystem level requirements typically under configuration 
control 

> Detailed design work either underway or has been completed 
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Fidelity Needed in Concept Design 

Solution 
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A Proposed Definition for “Feasible 




INCOSE 


iOI^ 

^f\ I 


• The term “feasible” is used frequently in concept design, but its 
use is often problematic 

> Often left undefined & subject to interpretation 

• This presentation uses “feasible” mission concept to mean: 

> Technical, cost, & schedule characteristics for a single, baseline mission 
concept design have been credibly converged to the 1st order by the end 
of Phase A, 

> such that the design may be developed, launched, operated, & 
decommissioned by a competent project team starting in Phase B within 
customary technical & programmatic margins 
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A Proposed Metric for Level of \ ncOSE 

Convergence (1 of 2) 

• Credible convergence to 1^^ order by end of Phase A means: 

> System level sizing & performance (SLSP) of mission elements is 
confidently determined to within 90% of SLSP when flight system is 
delivered 

□ For given cost & schedule constraints 

> i.e., there is residual uncertainty that SLSP could change by 
± -10% between end of Phase A & launch 
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A Proposed Model for Product Fidelity 
During Design Phases (Solid Black Curve)* 



Project Phase End 


Adapted from ref. 
(a). Fig. 3-4 


Figure 3 
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A Proposed Metric for Level of \ ncOSE 

Convergence (2 of 2) ^ ' 


• Solid black curve in Fig. 3 (uniform convergence) shows 
allowable SLSP error decreases as design moves from 
Phases A through C 

> End Phase A: order, or 90% (accurate to 1 digit, 10% error)* 

> End Phase B: 2^^ order, or 99% (accurate to 2 digits, - ± 1%> error) 

> End Phase C: 3^^ order, or 99.9% (accurate to 3 digits, ~ ±0.1%) error) 


• Metrics for SLSP error are approximate guidelines only 

> Coarse model that depicts an idealized trend of fidelity in each 
phase 

> Assume calculations done properly, but with incomplete or incorrect 
information / assumptions 


• * read as 9 x 10^ %, accurate to 1 significant digit 
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Mass 


INCOSE 


iOIS 


• For a 4,000 kg space observatory, system level mass should be 
known to: 


> End Phase A: 

> End Phase B: 

> End Phase C: 


Within ~ ± 10%, or - ± 400 kg of final launch mass 
Within - ± 1%, or - ± 40 kg of final launch mass 

Within - ± 0.1%, or - ± 4 kg of final launch mass 
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Role of (Selected) Resource Margins on INCOSE 

Required Convergence -Til ' 

• Solid black curve in Fig. 3 must be within envelope of required 
margins 

> Power & Dry Mass Margin requirements (per ref. (f)) are shown in Fig. 3 


□ End Phase A: > 25% 

□ End Phase B: > 20% 

□ End Phase C: > 15%> 
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Importance of Concept Design 


INCOSE 


Convergence to Project Manager 

• Project Manager at start of Phase B holds 25% margins for 
power & dry mass resources (Fig. 3) 

> Can accommodate concept design credibly converged to within 10% of 
flight sizing & performance values for power & dry mass 

□ Even if 10% error occurs in direction of needing more resources 

> Can’t accommodate concept design credibly converged to within 30% of 
flight sizing & performance values for power & dry mass 

□ if 30% error occurs in direction of needing more resources 

□ Design de-scope likely required 
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Techniques for Designing Mission Levei 

Trade Space 
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Concept Design Mission Levei Trade Space 

Selecting Trades to Expedite Convergence - 3 Cycle Example 




Figure 4 
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Concept Design Mission Levei Trade Space 

Selecting Trades to Expedite Convergence - 3 Cycle Example 
(Cont'd) 


• Approach in Fig. 4 deduces R&DC for C design by interpolating 
on results from A & B designs (bounding cases) 

> Technical capability of point C isn’t known at outset of study 


• More like root finding algorithm than like successive refinement 
design process typically used in Phases B & C 

> In Phases B &C, each design is refinement of “baseline” system level 
design from prior phase 

> In concept design process discussed here, typically there isn’t a 
“baseline” system level design until concept design is complete 
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Purposely views design problem from multiple perspectives 

> Illuminates aspects that otherwise may have remained hidden 


□ Helps stimulate creative thinking & mitigate biases 

□ Accelerates discovery of “unknown unknowns” 
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Why Selecting Bounding Cases is 
Important 


INCOSE 


• Failure to select bounding cases may cause extrapolation to 
determine R&DC for final solution 

> Adds risk in technical, cost, & schedule estimates 

> May result if both A & B designs exceed cost & schedule constraints 

□ Implies R&DC for B design didn’t identify “true” science or technology floor 
(presumes a solution exists) 

• Or, may cause need for more design cycles 

> Deadline may not permit, or may drive significant team overtime 

• Optimistic A designs & ‘"false” science floors for B designs are 
common 

> Customer’s vision often isn’t cost / schedule constrained 

> Customer may resist identifying “true” science or technology floor 

• Teams that recognize, or adapt to, these considerations 
pragmatically & quickly fare better than teams that 

LJi II livt/ioCji y 
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Selecting R&DC (Typical Case) 


INCOSE 


• Typical Approach 

> A Design: Most* parameters reflect realistic desired capability 

> B Design: Most* parameters reflect science or technology floor 

> C Design: Most* parameters are between A & B capabilities 

* but not necessarily all 

• R&DC for B design reevaluated after A design to assure solution 
space bounded 

> Presumes A design done first 

• Many parameters varied concurrently due to need to cover 
broad solution space in limited time** 

> Experience shows teams can sufficiently understand parameter 


sensitivities 

** after approach originally used by Mr. John Oberright, NASA / GSFC 
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Challenges in Determining Credible Design 

Convergence 
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Convergence Indicators Difficult to Define incOSE 
Objectively - ' 


• Concept design is inherentiy an expioratory process with 
reiativeiy high uncertainty 

• Concept design teams iearn at high rate 

> Early assumptions & conclusions may be invalidated by later findings or 
by unpredictable discovery of unknown unknowns 

• Yet, indicators are desired to heip avoid inferring convergence 
prematureiy, e.g., due to: 

> Insufficient rigor 

> Study funds or time being exhausted 

> Pressure to meet a milestone deliverable, etc. 

> Biases 
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Convergence Determinations Often 
Evident Only in Hindsight 


INCOSE 










Figure 5 
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Cost (or cost & schedule) estimates typically grow for 
designs A & B after accounting for knowledge gained in 
subsequent cycles 



Point Ag 
shows 
growth in 
cost & 
schedule 
estimates 
post B 
design 
cycle 


Cycle A is first 
design cycle 
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Points Ac & Be 
show growth in 
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estimates post C 
design cycle 
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Why 
Optimistic 


• A common characteristic of concept design is costs for a given 
design tend to increase with each design cycie 

> Particularly true for immature mission concepts that advance state of the 
art and that have high design uncertainty 


• As teams progress through cycies, they iearn more of what may 
have been omitted / incorrectiy assumed in prior cycies 

> After B cycle, cost of A design may increase for given technical capability 

> After C cycle, cost of A design may increase again, & cost of B design 
may increase 

□ Causes A & B points to move to right in Fig. 5 
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> When accompanied by schedule increases, A & B points also move into 
page 

> After C cycle, learning tapers off for most designs 

□ Sometimes a D cycle is needed, or may be planned from 
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Why 

Optimistic (Cont’d) 


• Cost analysis is normally performed using multiple methods 

> One method is “grass roots’’ - uses a work breakdown structure (WBS) 

• WBS dictionary for most space mission elements is relatively 
well known & largely existing, e.g., 

> Spacecraft, launch, ground systems, etc. 

• Conversely, WBS dictionary for new instruments is unique 

> Design dependent, evolves as instrument design evolves 

> Key aspect for designs dominated by new instruments 

• Multiple cost cycles typically needed to develop well understood 
WBS free of significant gaps & overlaps 

> Gaps common in design & cost in early cycles as team learns 

> Cost fidelity improves with understanding of both design and WBS 
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Subjective Criterion for Convergence 


INCOSE 


Determination - Significant Surprises ,3 

• One subjective criterion for credibie convergence is whether 
team has experienced significant surprises 

• Team that hasn’t experienced at ieast a few significant surprises 
shouid be cautious of its resuits 

• Lack of surprises may indicate: 

> Team hasn’t progressed sufficiently down learning curve 

> Team didn’t sufficiently exercise trade space or mitigate biases 

> Concept design study objective wasn’t sufficiently challenging 
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Recommended Practices 
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General Guidance 


INCOSE 


• Treat design cycles as precious resource 

> Essential, but in limited supply due to time & resources available 

> Focus team efforts on developing product, omit peripheral tasks 

• Don't retrofit A & B designs with insights from later cycles 

> Time better spent just applying learning to final design 

• Don't let first cost estimate be final cost estimate 

> Be cautious of early results, they may not be as initially appear 

• Document design results in reports at end of each cycle 

• Maintain 1^^ order analysis depth in concept design 

• Avoid significant rounding errors 

• Recognize typical phases of concept design 
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Maintain 
in Concept Design: Anaiogy 


• Pre-Phase A & Phase A teams evaluate multiple designs in 
broad trade space in relatively short period 

> Analysis tools used typically are 1st order precision, agile enough to 
adapt to frequent & significant system level changes 
m Analogy: “Hacksaw” 


• By comparison, analysis tools typically used in: 

> Phase B are 2 ^^ order precision; assume system level design stable 
U Analogy: “File” 


> Phase C are 3''^ order precision; assume both system & subsystem level 
designs stable 

□ Analogy: “Polisher” 
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Maintain 

in Concept Design: Anaiogy (Cont’d) 


• Team using “hacksaw” in Phase C has done something wrong 

> Didn’t credibly converge order solution by end of Phase A 

> Re-doing system level concept design work late & out of sequence 


• Team using “poiisher” in Phase A is doing something wrong 

> Won’t move quickly or broadly enough to rough-out & credibly converge 
order solution* 

□ Recognize some design elements may not even exist in final concept design 


Some high risk eiements may seiectiveiy warrant added scrutiny 
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Avoid Significant Rounding Errors 


INCOSE 


• Rounding errors can significantly affect margin determination if 
team doesn’t use sufficient numerical safeguards 

> In some cases, rounding errors can fully mask margins such as those for 
mass & power shown in Fig. 3 

• To avoid masking resource margins, bookkeep design & 
performance calculations to 3 significant digits & report out to 
2 significant digits* 

> Should not be taken to imply there is 3-digit accuracy in concept design 
work - there usually is not 

> Simply a numerical safeguard to avoid propagating rounding errors that 
could overwhelm ability to adequately determine design or performance 
margins 
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* as a minimum guideiine 
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Recognize Four Unofficial, but Typical, INCOSE 
Phases of Concept Design , - 


• Concept design teams developing immature mission concepts 
that advance state of the art often experience four phases of 
work 

> 1) Unbridled Optimism 

> 2) Shock 

> 3) Denial 

> 4) Acceptance 

• The quicker a team moves through phases 1,2, & 3 and arrives 
at Phase 4, the better that team will fare 
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Closing Thoughts 
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Closing Thoughts 


INCOSE 


• Concept design phases have extraordinary ieverage over 
project success 

> There is a business case to conduct in rigorous & pragmatic fashion 

□ Particularly for immature mission concepts that advance state of the art and 
that have high design uncertainty 

> Provide unique venue to expiore & converge system ievel design 

• Done weiij concept design can provide executabie system ievei 
design baseiine for project teams in Phase B & iater phases 

• Not done weii, some work of concept design phases usuaiiy wiii 
have to be done again 

> The later this realization occurs, the more expensive the resulting 
redesign is likely to be 
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Questions ? 
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Effect of Rounding Errors on Margin I N COS E 

Determination: Exam pie ' 


Case 1: Power Available 

Max. Estimated Power Required 

Power Margin = 100 (200 W - 249 W)/ 249 W 


= 200 W 
= 249 W 
= -19.7% 


Case 2: Power Available 

Max. Estimated Power Required 

Power Margin = 100 (200 W- 151 W)/ 151 W 


= 200 W 
= 151 W 
= 32.5% 


The margins for Cases 1 and 2 are -19.7%> and +32.5%>, respectively 

Now consider a third case In which a designer rounds calculations to the 
1st digit In Cases 1 and 2 

Case 3: Power Available = 2x 10^ W 

Max. Estimated Power Required = 2x 10^ W 

Power Margin = 100 (2 x 10^ W- 2 x 10^ W) / 2 x 10^ W = 0% 


The margin for Case 3 Is 0% 
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Effect of Rounding Errors on Margin 


INCOSE 


Determination: Exam pie (Cont’d) 

• Required power margin at end of pre-Phase A is 30% (Fig. 3) 

> Comparing Case 3 to Case 2 shows how rounding to 1st digit can fully 
mask a margin of over 30% 

> Additional errors can accrue when combinations of rounded results are 
used in successive calculations 

• To avoid masking resource margins, bookkeep design & 
performance caicuiations to 3 significant digits & report out to 
2 significant digits* 


• Note: 

> Margin calculation method is per ref (f), Table 1.06 


* 


as a minimum guideiine 
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Benefit of Study Phase Investment 

Ref. (a), Fig. 2-1 (Dec 1992) 
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Documenting 
in Reports at End of Each Design Cycie 


• Provides official study record of what team did, how team 
did it, & what team found for present (& future) team use 

• Reports are developed for each subsystem / discipline 

> Built from standardized templates 

□ Include analysis methods & example calculations 

> Provide coherent technical waypoints that enable team to recall 
designs & performance from prior cycles 

□ Often needed for scaling or comparison 

□ High rate of design changes makes recollection difficult otherwise 

> Used for system level review, subsystem integration, independent 
review, new / follow-on team member orientation 

• Once approved, reports typically are under informal 
configuration control of Mission Systems Engineer 

> Briefings can be generated quickly from approved reports 

^ . r. . . , ■ r X- ■ f -L anniversary 

> Briefings contain only information in approved reports mnuai incose 
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Role of (Selected) Resource Margins on INCOSE 

Required Convergence ’ 

• Solid black curve in Fig. 3 must be within envelope of required 
margins 

> Power & Dry Mass Margin requirements (per ref. (f)) are shown in Fig. 3 

□ End Phase A: > 25% 

□ End Phase B: > 20% 

□ End Phase C: > 15%> 

• Cost (not shown in Fig. 3) serves as design constraint 

> Cost margin (per ref (g)) 

□ Cost through Phase D: > 30%> (guideline at Phase B start) 

□ Cost through Phase D: > 25%> (reouirement at Phase C start) 

• Other programmatic margin requirements apply as well, e.g., 

> Schedule margin (per ref (g)), not shown in Fig. 3 
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Recognize Typical (but Unofficial) Phases in COSE 
of Concept Design ' 


• Concept design teams developing new designs that advance 
state of the art often experience four phases of work 

1) Unbridled Optimism 

> This phase features unbridled, optimistic performance desires levied as 
“requirements’’ before team gains credible understanding of associated 
cost & schedule 

> Meetings often not well-focused on study objectives 

□ Instead, feature extended advocacy discussions (e.g., why mission has best 
science of all competing missions, why it has best chance to win, etc.) 

2) Shock 

> This brief phase usually begins after team completes its first credible cost 
estimate 
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Recognize Typical (but Unofficial) Phases in COSE 
of Concept Design ( Cont’d) ' - ’ 


3) Denial 

> This phase features abundant rationalizations as to why models used to 
estimate costs weren’t representative 

> Team points to any aspect of mission - except excessively high technical 
capability - as reason costs are too high, so science return remains 
compelling relative to competition 

4) Acceptance 

> This phase features ultimate realization technical capability / science 
return must be lowered to design a credible mission concept 

□ One that meets cost & schedule constraints according to established 
independent review standards 
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